Revalidation for occupational physicians: problems or promise?

In an editorial in this Journal I asked the question whether recertification, albeit for specialists, would be far behind. Over the last few weeks, most doctors in the United Kingdom received a summary paper concerning the General Medical Council's (GMC) proposals for revalidation. In essence, the GMC is proposing the submission of a portfolio on a five yearly basis. The events and scandals that have come to light recently regarding the medical profession have ensured that re-certification is extended to one of revalidation rather than mere specialty re-certification. Portfolios are not new and have long been an integral part of the graphic arts in the tracking of the professional development of artists over a period of time. They have been used in education and now medical education as a means of encouraging professional growth. They were thought to have potential in clinical performance assessment over a period of time, creating a means of authentic assessment, that is to say assessment that looks at performance and practical application of theory. Medical education over the last decade has been occupied with the search for learning strategies that promote adult, learner-centred ideas in the belief that these will stimulate 'deep' learning processes. These processes impact on behaviour, intuition and performance of the physicians. Educational programmes have long been thought to be more effective when they are interactive, include reflective components and are related to experience. These components lead to increased autonomy for participants and to independent learning. Portfolios have developed as one response to these ideas, which are precisely those behaviours upon which medical education generally rests and revalidation specifically will attempt to elicit from registered physicians. Portfolios have long been promoted in general practice at both vocational and higher professional training levels. Others argue persuasively for the adoption of experience-based and workplace learning in continuing medical education, for which portfolios are an efficient tracking mechanism. Whilst others admit their effectiveness in stimulating reflective learning, other authors highlight difficulties in their use for assessment purposes. The individual physician may keep quite different material in their portfolios if they are used for formal assessment purposes as opposed to learning thus the GMC concept of the revalidation folder may achieve neither an educational benefit nor a reasonable means of performance assessment. Indeed the concept of performance assessment by this means is so alien that very clear guidance will be needed on the type of evidence to be furnished by physicians, otherwise a risk of inadequate material being presented for the intended purpose, will become very real indeed. This is particularly important if lay people are likely to be involved in the process. The use of portfolios in formal assessments are known to have negative impacts on the material collected. Left to their own devices, physicians are unlikely to place incidents that have not been well managed in their portfolio. Such critical incidents are undoubtedly a rich source of learning and have long been a part of educational methodology. They give insight into the development of any physician, irrespective of specialty and seniority. It is understandable that physicians would be reluctant to allow such material to be allowed for such purposes as they have long become accustomed to the traditional approach to occupational medicine in particular, namely a competitive examination that seeks excellence. The material they contain is personalised with but a few points of objectivity for comparison between occupational physicians or indeed to establish standards. In occupational medicine, differences in practice settings may complicate this process even more. Can one really make a reasonable assessment of an occupational physician who may, on the one hand be working in a private sector organisation with a budget of over £2 million compared to the cash-strapped public sector organisation or a smallor medium-sized enterprise with a mere 200 employees? Even more so, there are considerable difficulties in equating the work of an occupational physician with that of a hospital doctor or general practitioner. Indeed, how does one make the judgement on performance, without knowledge of the workplace or the organisational dynamics in which the occupational physician must practise to his or her professional standards, which often have varying degrees of acceptance by their employers? The problem list gets longer. Assessment is also labour intensive requiring careful reading and response to the portfolio owners, learning objectives and evidence of whether or not they have been met. Portfolios undoubtedly can be effective in support and facilitation of personal learning and professional growth but cumbersome in comparative assessments even when this is against a notional standard. Little research is available in the literature on a technical quality of large-scale portfolio assessments and what has been done, has been largely within the system. One study provided a six-point scale from 'exceptional achievement' to 'minimal evidence of achievement'. Descriptors were provided under each category with a variety of headings including: focus/organisation, development and mechanics. Rigorous field testing demonstrated the validity and reliability of the scale but the correlation patterns remained unclear. The need for new methods of assessment in the context of new ideas on performance assessment is clear. The authors concluded that learning opportunities are maximised when portfolios invite individual ownership. They also have a huge potential for assessment in the context of adult and self-directed ideas about learning. The concept of portfolios however, implies students selected, nonstandardised work. Hence, the present reliance of assessment on notions of validity and measurement will be challenged by our dependence on this means of performance assessment. The effective assessment of portfolios will continue to be difficult if we remain trapped in this traditional view of assessment. Portfolios which are non-standardised will not readily fit with assessment based on comparing physicians one with another, or indeed with the issuing of grades or marks.